Integrated development and on-the-fly simulation of multimodal dialogs

This paper presents a tool that allows the integrated development of multimodal dialog systems. We show how graphical/haptical and speech interface can be designed in an integrated way, which is in contrast to many state-of-the-art HMI design processes. Graphical/haptical and speech interface are often designed more or less independently of each other, which can result in inconsistencies of various kinds. The integrated design results in truly multimodal interfaces, which allow a change in modality at any point. A key feature is the capability to immediately simulate the complete HMI during the specification process, which allows evaluation of all modalities in combination at very early development stages. The tool can be used for rapid prototyping of multimodal dialog systems in the research area, but at the same time fulfils several requirements that are posed to the HMI development in a mass production process, e.g. in the automotive area.